1. Technical Field
This invention relates to the computer processing and display of digital, multispectral imagery for the purpose of identifying ground targets and classifying the imagery to create thematic maps.
2. Description of Prior Art
Ground targets in imagery have been identified commonly through the subjective process of an art referred to as photointerpretation. This process relies on shapes, tones, textures, colors, and associations in the image to infer the nature of the ground feature. The results are highly correlated with the skill and experience of the photointerpreter.
Multispectral imagery is a collection of coregistered images (typically  less than 23), each image being collected in a different, broad region of the electromagnetic spectrum commonly called a band. These bands are collected nominally between 0.400 xcexcm (micrometers) and 2.500 xcexcm wavelength; this region is generally referred to as the visible/infrared-red (VIR) region. The computer classification of multi-spectral imagery, to create thematic maps, has usually been of two general types, referred to as supervised and unsupervised.
In the case of supervised classification, regions within the image, that are believed to represent relatively homogeneous clusters of pixels (picture elements) that are characteristic of a particular type of ground cover of interest, are selected to be what are known as training sites. The selection can and most often does proceed by an experienced photointerpreter marking the boundaries of the training site with an on-screen cursor controlled with some pointing device such as a mouse, trackball, joystick, lightpen, or keyboard arrow-keys. The selection of particular training-site locations within an image, and their resultant shapes, is subjectively determined by an operator. The selection is based on their interpretation of the image and possibly is guided by information called xe2x80x9cground truthxe2x80x9d obtained from on-site reconnaissance on the ground.
These training sites are then used by various computer algorithms, again commonly selected by an operator from a plurality of choices, and typically statistical in nature, (such as the Maximum Likelihood classifier) to sequentially test the spectral characteristics of every pixel in the image against the pixels in the selected training sites. The tests performed are for a similarity metric for each pixel with a plurality of multispectral bands that may represent all bands recorded or a sub-set selected by the operator. The definition of the agreement or similarity varies with the algorithm selected. The intent is to categorize or classify every pixel as being most like the pixels in a particular training site, when considered from the perspective of the n-dimensional feature space defined by the number of bands selected for the comparison.
In the unsupervised classification approach, an operator selects a particular algorithm from a plurality of choices commonly provided by the manufacturer of the image processing software. That algorithm then performs iterative comparisons in n-dimensional space, where n is the number of multispectral bands chosen by the operator to be considered. The comparisons result in every pixel being assigned to a cluster of spectrally similar pixels. If the statistical characteristics of any or all of the existing clusters exceed certain thresholds, either hardcoded by the software manufacturer or selected by the operator, clusters can be split or lumped together. After a pre-selected number of iterations, or alternatively a pre-selected number of clusters is attained, the program halts and displays the spectrally dissimilar clusters, wherein the clusters are commonly pseudo-colored to help in their visual differentiation. The operator must then subjectively decide what every cluster represents in terms of a ground target or type of ground cover. This is again accomplished through either photointerpretation procedures, or the collection of ground-truth from onsite inspection, or a combination of both.
Numerous band-ratios and indices, utilizing two bands, have been in use for years to accentuate spectral features. Thresholding a band-ratio (or vegetation index) can provide a binarized (two-level) classification of the images. Where to establish the threshold is subjective.
It can be seen that the subjective judgement of the operator is extremely important in obtaining accurate results for all the methods discussed above. Their experience is extremely important in making the subjective decisions. In addition, the availability of accurate ground truth is important in achieving good classification results. Although the algorithmic processing of the imagery can be performed for an n-dimensional feature space where n is greater than 1, the operator is constrained to only being able to see the effects of three bands at any one time. This affects the ability to either correctly select homogeneous training sites or properly identify the corresponding type of ground cover for a particular cluster-class derived from unsupervised classification. One does not know a priori what the best three multispectral bands are to help identify the various types of ground cover in the image.
An additional problem is that one can expect that every algorithm chosen will result in a classification of the ground cover that is different from the results of every other algorithm. The initial selection of an appropriate algorithm from the plurality of choices available will be influenced by the experience of the operator. The acceptance or rejection of the classification results will also be a subjective decision made by the operator. While there are objective tests of the accuracy of classification, even the interpretation of the tests is ultimately subjective.
A common problem in multispectral classification, particularly with high spatial-resolution imagery, is the incorporation and ambiguous identification of a shadow class. Related to this is, if the operator does not recognize the existence of one or more spectrally unique types of ground cover, those classes will be relegated to an xe2x80x98unknownxe2x80x99 or xe2x80x98undefinedxe2x80x99 class with supervised classification.
Another approach for creating thematic maps, which has evolved in recent years, is an extension of multispectral remote sensing. If a sufficient number of as discrete bands of the electromagnetic spectrum are sampled, the resulting plot of the apparent reflectance of a ground target approaches a continuous, smooth curve. The imaging instruments that collect a very large number of bands, usually between about 26to 28 narrow bands, nominally between 0.400 xcexcm and 2.500 xcexcm wavelength, are called hyperspectral sensors. The advantage of hyperspectral imagery is that, with careful calibration, correction of illumination variations, and compensation for atmospheric absorption and scattering effects, an apparent reflectance can be derived from the radiance values for every pixel; then the pixels can be compared to a library of laboratory-derived reflectance spectra for a match. Thereby, ideally, the need for subjective judgement on the part of the operator is eliminated. However, the atmospheric corrections are not a trivial undertaking, the available spectral libraries are reasonably complete for minerals only, and the hyperspectral sensors are not extensively deployed as yet. The analysis and subsequent processing of hyperspectral imagery to create thematic maps is generally more difficult and requires a higher level of skill and experience than for multispectral imagery.
Multispectral (and hyperspectral) images are invariably displayed as a single false-color image created from a combination of three particular bands out of the total number of bands recorded. However, most remote sensing image processing software currently available only provides the ability to plot scattergrams from two bands simultaneously.
A ternary diagram (a.k.a. triangular plot) is a graphical plot based on the use of an equilateral triangle; it is used for displaying the relationship between three variables that are related in a manner such that they sum to unity or 100 percent, For plotting purposes, the triangle is internally subdivided into three sets of parallel lines, each set being parallel to one of the sides of the triangle. The lines, which may be hidden for clarity, represent percentages of each of the three components being plotted. These three variables are commonly referred to as end-members. Each apex of the triangle represents 100 percent of an end-member and the side of the triangle opposite a particular apex represents zero percent. A point in the center of the triangle represents equal proportions (⅓) of all three end-members. Ternary diagrams are tools frequently used in geology, metallurgy, and physical chemistry. A familiar use is in classifying soils by texture. In this application, the end-members are sand, silt, and clay. Dividing lines on the ternary diagram define boundaries for soil textural names, such as xe2x80x9cloam.xe2x80x9d Ternary diagrams are exceedingly tedious to plot by hand and are subject to plotting error.
The U.S. Geological Survey Flagstaff Field Center wrote a computer program in 1980 called Mini Image Processing System (MIPS). It is available to the public for VAX and Unix operating systems. MIPS contains a module, called xe2x80x9cTriplot,xe2x80x9d for displaying triangular plots of multispectral imagery. However, there is no indication that there is any awareness among the developers that particular band selections result in clusters of points that are separable into classes of materials. The documentation simply says, xe2x80x9cUses: If three multispectral images have been converted to reflectance, then the percentage of the total reflectance represented by each can be seen in the plot.xe2x80x9d MIPS also contains a module, called xe2x80x9cReflect,xe2x80x9d for converting raw data into ground reflectance.
Research Systems Inc. (Boulder, CO) markets and distributes a remote sensing software package written by Better Solutions Consulting LLC (Boulder, CO) called ENVI. It has a feature, that they call xe2x80x9cdancing pixels,xe2x80x9d that links an image to a 2-D scatterplot (Cartesian coordinates). Pixels selected within a region of interest in the image are plotted on an orthogonal-axis scatterplot and, conversely, points selected in the scatterplot cause their corresponding pixels in the image to be highlighted. Again, however, there seems to be no awareness on the part of the developers that one can have any a priori knowledge about the identity association of the points in the scatterplot, or awareness of the value of any particular band combinations.
Accordingly, besides the objects and advantages of multispectral classification as described in xe2x80x9cDescription of Prior Artxe2x80x9d above, several objects and advantages of my invention, hereinafter referred to as xe2x80x9cTernary Multispectral Analysis,xe2x80x9d are as follows:
(1) To provide a means to identify ground reflectors without need for:
(a) a highly skilled photointerpreter;
(b) ground-truth;
(c) more than three, broad bands available on the Landsat Thematic Mapper (TM) or similar multispectral sensors;
(2) To provide a means to locate spectral targets defined as particular types of ground cover;
(3) To provide a means to interactively classify multispectral imagery and thus generate thematic maps of ground cover without need for:
(a) a highly skilled photointerpreter;
(b) ground truth;
(c) more than three TM bands;
(4) To provide a means to automatically classify multispectral imagery and thus generate a thematic map of ground cover without the need for:
(a) an operator;
(b) photointerpreter;
(c) ground truth;
(d) or more than three TM bands;
(5) To provide a means to minimize the impact of the xe2x80x98shadow classxe2x80x99 through an inherent normalization;
(6) To provide a means to estimate the proportion of end-members in a mixed-pixel;
(7) To provide a means to estimate turbidity and type of sediment in water;
(8) To provide a means to generate false-color imagery with constant colors for the same type of reflectors, and;
(9) To provide a means to interactively accomplish empirical corrections of atmospheric scattering and absorption for a multitude of multispectral bands.
Still further objects and advantages will become apparent from a consideration of the ensuing description and accompanying drawings.
The method and process of Ternary Multispectral Analysis disclosed herein includes image display, graphical visualization, image processing, mathematical techniques, and signal processing.
In accordance with the present invention, plotting of pixel values from multispectral images from appropriate Landsat TM (Thematic Mapper) and ETM+ (Enhanced Thematic Mapper Plus) bands (or their equivalents) on a ternary diagram results in a feature space display that effectively clusters vegetation separately from minerals. Further, minerals are separated as a cluster along a major symmetry-axis of the triangle and have lesser dispersion at approximately right angles to the general elongation of the mineral cluster. Additionally, various kinds of vegetation are dispersed along a second major linear cluster, separated from the mineral cluster.
According to one feature of the invention, a ternary diagram, derived from spectral reflectance libraries convolved to the bandwidths of particular bands from an applicable multispectral sensor system (e.g. Landsat TM), will display point clusters defining separable regions of a plurality of landcover themes.
According to another feature of the invention, a computer-graphics ternary diagram has a real-time link to a computer-displayed image, through a lookup table. This allows selected pixels in the image to be identified as particular points in the ternary diagram and, conversely, pixels corresponding to selected points in the ternary diagram can be highlighted in the image. This reciprocal functionality allows the identification of targets in the image and/or the generation of spectrally-derived thematic maps of groundcover.
According to another feature of the invention, schemas can be constructed to automatically classify an image into themes. The schemas can be customized for particular goals.
In still another feature of the invention, ternary diagrams can be used to empirically correct for variations in solar illumination and atmospheric absorption and scattering.
Other objects, features, and advantages will be readily apparent.